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A LAGRANGIAN APPROACH TO WEAKLY COUPLED 
HAMILTON-JACOBI SYSTEMS 

H. MITAKE, A. SICONOLFI, H. V. TRAN, AND N. YAMADA 


Abstract. We study a class of weakly coupled Hamilton-Jacobi systems with a specific 
aim to perform a qualitative analysis in the spirit of weak KAM theory. Our main 
achievement is the definition of a family of related action functionals containing the 
Lagrangians obtained by duality from the Hamiltonians of the system. We use them to 
characterize, by means of a suitable estimate, all the subsolutions of the system, and 
to explicitly represent some subsolutions enjoying an additional maximality property. A 
crucial step for our analysis is to put the problem in a suitable random frame. Only 
some basic knowledge of measure theory is required, and the presentation is accessible 
to readers without background in probability. 


1. Introduction 


This paper deals with weakly coupled Hamilton-Jacobi systems of the form 


H\(x, Dui) + A 1 • u = a 
Hm{x, Dum ) + A M • u = a 


(HJa) 


on the flat torus T A . Here u = (ui, • • • , um) is the vector valued unknown function, Dm 
the gradient of a a real number, and Hi are mutually unrelated convex Hamiltonians 
enjoying standard additional properties (see Section 2). The A* are the rows of the so 
called M x M coupling matrix A := (A 1 • • • A A/ ), which constitutes the relevant item in 
the problem. 

We are specifically interested in the setting which should correspond in the scalar case, 
namely when M = 1 and A is just a constant, to taking A = 0. Then the system reduces 
to a single equation on T jV not directly depending on the unknown and classified as of 
Eikonal type. 


In this framework a rich qualitative theory has been developed by linking PDE facts 
to geometrical/dynamical properties. Representation formulae for (sub)solutions have 
been provided through minimization of a suitable action functional, showing, among other 
things, the existence of an unique value of a, named a critical value, for which (viscosity) 
solutions do exist. This material has found applications in a variety of related asymptotic 
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problems, and connections with Hamiltonian dynamics have been furthermore established, 
at least when the Hamiltonian is sufficiently regular. This body of results is a part of the 
so-called weak KAM theory, see [1, 4, 6, 7, 8, 11] for details. 

We recall that if instead A > 0 the corresponding equation can be uniquely solved on the 
whole torus for any a and the solution is the value function of a related control problem 
with A playing the role of discount factor. 

To find an analogue of the Eikonal case for systems, it is convenient to start from paper 
[10], where the class of monotone systems is introduced, and existence and uniqueness 
results of (viscosity) solutions are established. Regarding our system, to be a monotone 
one corresponds to the following conditions on the coupling matrix: 

• any non-diagonal entry of A is nonpositive; 

• A is diagonal dominant, namely ^2jLi A ij > 0 for any i G {1,...,M}; 

• strict diagonal dominance holds at least for one row. 

This setting should be then analogized to strict positiveness in the scalar case and in 
this perspective it is consistent to focus on the limit setup where A satisfies: 

• any non-diagonal entry of A is nonpositive; 

• any row sums to 0. 

It has been actually a merit of [3, 16, 17, 18] to have first realized and pointed out 
that under the above assumptions on the coupling matrix, some phenomena, already 
occurring in the Eikonal scalar case, also take place for systems, and can be analyzed 
in the spirit of the weak KAM theory. In these papers it has been in particular showed 
the existence of a critical value as the minimal value for which the corresponding system 
admits subsolutions, and some related asymptotic problems have been studied providing 
generalization of results already known in the scalar case. Control interpretation for the 
Hamilton-Jacobi system has clearly been investigated in [17, 18]. We also refer to [9] for 
the study of the weak KAM theorem of another type of systems. 

A significant step forward in this direction has been more recently performed in [5], 
proving that, similarly to what happens in the scalar case, a distinguished subset of the 
torus, named after Aubry, can be defined with the crucial property that the maximal 
critical subsolution (i.e., a subsolution to the system with a equal to the critical value) 
taking a given value, among admissible ones, at any fixed point of the Aubry set is indeed 
a critical solution. The aforementioned admissibility refers to the fact that there is a 
restriction in the values that a subsolution of the system can assume at any given point. 
This is a further relevant property pointed out in [5], which genuinely depends on the 
vectorial structure of the problem and has no equivalent in the scalar case. 

All the above results, even if of clear interest, however pertain to the PDE side of the 
theory, and are solely obtained by means of PDE techniques. The geometric counterpart 
is so far missed and the intertwining between PDE and dynamical aspects, which is at the 
core of the weak KAM theory, has consequently still to be understood in the framework 
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of systems. This is actually the primary task the paper is centered upon, and is above all 
performed by putting the problem in a suitable random frame. 

As a first step we consider all the possible switchings between indices {1, • • • , M} of 
the system on an infinite time horizon. This gives rise to the space of {1, • • • ,M}-valued 
cadlag paths, denoted by V, endowed with the Skorohod metric and the corresponding 
Borel u-algebra J~. The coupling matrix, being under our assumptions generator of a 
semigroup of stochastic matrices, induces a linear correspondence between the simplex of 
probability vectors of , i.e., with nonnegative components summing to 1, and a simplex 
of .F-probability measures on V, see Subsection 3.1. 

This construction is indeed equivalent to that of a Markov chain with rate matrix —A, 
and in fact key formula (3.1) defining the family of probability measures is nothing but 
the usual finite-dimensional distribution formula with given initial distribution. However 
we would like to emphasize that the advantage of our approach is to avoid introducing an 
abstract probability space, we just work with concrete path spaces, and also avoid explicitly 
using notions as stochastic process, conditional probability and other probabilistic tools. 
This makes the presentation self-contained. 

We make corresponding to elements of V R A -valued cadlag velocity paths and obtain 
by integration of it the admissible random curves on T N , see Subsection 3.3. Action func¬ 
tionals are then obtained by averaging, with respect to previously introduced probability 
measures on T>, line integrals over random curves on time random intervals of the La- 
grangians given by duality by the Hamiltonians of the system, see (4.1), which justifies 
the title of the paper. 

The effectiveness of our approach is demonstrated by recovering some crucial facts of 
the scalar case. Namely, we fully characterize all subsolutions of the system, for any a 
greater than or equal to the critical value, as the functions from T N to satisfying a 
suitable estimate with respect to our action functionals, see Section 4 and Theorem 5.7. 
We moreover use the action functionals to represent explicitly critical and supercritical 
subsolutions enjoying an additional maximality property, through a suitable minimization 
procedure, see Theorem 5.2, and to give a dynamical formulation of the property of being 
admissible for a value at a given point, see Theorem 5.5. By this way we also provide a 
representation formula for critical solutions taking a prescribed admissible value at a given 
point of the Aubry set, complementing the result of [5], see Theorem 5.6. 

The paper is organized as follows: in Section 2 we set forth the problem and recall 
some known facts about critical/supercritical subsolutions and the Aubry set. Section 3 
is devoted to illustrate the random frame in which our qualitative analysis takes place: 
the family of probability measures P a , for any probability vector a of M m , is introduced 
and key notions as admissible control and stopping time are given. In Section 4 we define 
the action functionals and prove the fundamental estimate for subsolution to the system. 
Section 5 is about representation formulae for subsolutions and related results. Finally the 
two appendices gather basic material on stochastic matrices and spaces of cadlag paths. 
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2. Setting of the problem 

Here we introduce the system, which is the object of investigation, as well as standing 
assumptions and basic preliminary facts. We refer to [3, 5, 16, 17] for proofs and more 
details on the results stated. 

As already pointed out in Introduction, we will be interested on the one-parameter 
family of systems (HJa) 


H\{x, Du \) + A 1 • u = a 
Hm(x, Dum) + A M • u = a 


(HJa) 


posed on the flat torus T N identified to /7, N . Here u = (rq, • • • ,um ) is the vector¬ 
valued unknown function, A* are the vectors given by the rows of the M x M coupling 
matrix A, and a varies in R. The following conditions will be assumed throughout the 
paper without any further mentioning. On Hamiltonians Hi we require 

(HI) Hi is continuous in both variables; 

(H2) Hi is convex in p\ 

(H3) Hi is super linear in p; 


The growth condition in (H3), together with (HI), (H2), allows defining the correspond¬ 
ing Lagrangians via the Legendre-Fenc.hel transform, namely 

Li(x, q) = max (p ■ q — HAx,p)) for any i, 
peR™ 

and they inherit from Hi the properties of being continuous, convex and super linear at 
infinity. 

We furthermore require on coupling matrix A: 

(H4) any non-diagonal entry of A is nonpositive. 

(H5) any row of A sums to 0. 

(H6) A is irreducible. 

Irreducible means that, given any nonempty subset of indices I C {1, - - - ,M}, there is 
i £ I, j ^ I with A ij ^ 0; loosely speaking this condition means that the system cannot 
be split in separated subsystems. 

As made precise in Appendix A, the key point is that (H4), (H5) are equivalent to —A 
being generator of a semigroup of stochastic matrices. We also recall that under (H4), 
(H5), (H6) the matrix A is singular with rank M — 1 and kernel spanned by 1, namely 
the vector with all components equal to 1, moreover irn(A) cannot contain vectors with 
strictly positive or negative components. This in particular implies irn(A) nker(A) = {0}. 
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2.1. Notation. The projection of l w onto T A = induces a structure of additive 

group on T N . To ease notations we will indicate throughout the paper by the usual 
symbols +, — the corresponding operations between elements of the torus. 

The notion of viscosity (sub/super)solution can be easily adapted to systems as (HJa), 
we will drop in the following the term viscosity since no other kind of weak solution will 
be considered. 

2.2. Definition. A continuous function u = (ui,--- ,um ) is a subsolution (resp., super¬ 
solution) of (HJa) if the inequality 

Hi(x, Dif(x)) + A* • u(x) <a (resp., > a) 

holds for every x £ T iV , i £ {1, • • • ,M}, and if £ C 1 (T r ') such that Ui — if attains a 
maximum (resp., minimum) at x. We call u a solution if it is both a subsolution and 
supersolution. 

2.3. Remark. One could wonder why we are considering systems with the same constant 
appearing in the right-hand side of any equation, while a more natural condition should 
be to have instead a vector of M m , say a, with possibly different components. We point 
out that, under our assumptions, such a setting is actually no more general. In fact, if 
we write the vector a as ai + a 2 with ai = a 1 £ ker(A), a 2 £ im(A), where this form is 
uniquely determined because im(A) flker(A) = {0}, and pick b with Ab = —a 2 , then u is 
a (super/sub)solution to (HJa) if and only if u + b satisfies the same properties for the 
system obtained from (HJa) by replacing in the right hand side a 1 by a. 

2.4. Remark. Due to the coercivity condition, any subsolution to (HJa) is Lipschitz 
continuous. Moreover, owing to the convexity of the Hamiltonians, the notion of viscosity 
and a.e. subsolutions are equivalent for (HJa). Furthermore, we can express the same 
property using generalized gradients of any component in the sense of Clarke. Namely, w 
is a subsolution to (HJa) if and only if 

Hi(x,p) + A* • w(.x) < a 

for any x £ T iV , p £ diVi(x), i € {1, - - • , M}, where dwi(x) indicates the generalized 
gradient of Wi at x. 

Here are two basic propositions. 

2.5. Proposition. The family of all subsolutions to (HJa), if nonempty, is equi-Lipschitz 
continuous with Lipschitz constant denoted by i a . 

2.6. Proposition. The family of subsolutions to (HJa) taking the same value at a given 
point, if nonempty, admits a maximal element. 
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We define the critical value 7 as 

7 = infja £ R | (HJa) admits subsolutions} 

The infimum in the definition of 7 is actually a minimum, as made precise below. 

2.7. Proposition. The critical system (HJ 7 ) is the unique in the one-parameter family 
(HJa), a € R, for which there are solutions. 

Following [5], we give the definition of the Aubry set A C from the PDE point of 
view: 

2.8. Definition. A point y belongs to the Aubry set A if any maximal critical subsolution 
taking a given value at y is a solution to (HJ 7 ). 

Roughly speaking the Aubry set, which is a closed nonempty subset of T N , is the 
place where it is concentrated the obstruction in getting subsolutions of system below 
the critical level. More specifically, there cannot be any critical subsolution which is, in 
addition, locally strict at a point in A, in the sense of the above definition. 

2.9. Definition. For a given critical subsolution u, a component Ui, for some i € {1, • • • , M}, 
is said locally strict at a point y £ if there is a neighborhood U of y and a positive 
constant 6 with 

Hi(x, Duf) + A* • u < 7 — 8 a.e. x £ U. 

In analogy with the scalar case, we have a following property: 

2.10. Proposition ([5, Proposition 3.9]). A point y A if and only if for any given index 
i £ {1, * • • , M}. there exists a critical subsolution u with Ui locally strict at y. 

An interesting fact pointed out in [5] is that there is a restriction on the values that a 
subsolution to (HJa) can attain at a given point. This is a property due to the vectorial 
structure of the problem and has no counterpart in the scalar case. The authors refer to 
it as rigidity property or rigidity phenomenon. For a > 7 , we define for x £ 

F a (x) = {b £ R m I 3 u subsolution to (HJa) with u(x) = b}. (2-1) 

Notice that F a {x) is convex because of the convex character of the Hamiltonians, in 
addition, if b £ F a (x) then b + y 1 is still in F a (x) for any y £ R, being 1 £ ker(A). This 
is in a sense equivalent of adding a constant to a subsolution in the scalar case. We have 
a following rigidity phenomenon on A: 

2.11. Proposition ([5, Theorem 5.1]). The admissible values for critical subsolutions at 
a given point in A are of the form 

b + y 1 

where b £ R M depending on y, and y £ R. 
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3. Random setting 

3.1. A family of probability measures. To build up the random frame appropriate for 
systems, we introduce a family of probability measures defined on V, namely the space of 
cadlag paths taking values in {1, • • • , M} endowed with the cr-algebra J 7 , see Appendix B. 
Averaging with respect to such measures will play a crucial role in the subsequent analysis. 
We will more precisely show that the coupling matrix A induces a correspondence between 
the simplex S of probability vectors of , and a simplex of probability measures on V. 

It is convenient for later use to start by recalling that the family of cylinders of J 7 , or 
of J~t for any t > 0, is a semi-ring. Namely it contains the empty set, is closed by finite 
intersections, and the difference of two cylinders is a finite disjoint union of cylinders. 
Therefore, taking into account that J 7 , Ft are generated by cylinders, we get by the 
Approximation Theorem for Measures, see [13, Theorem 1.65]. 

3.1. Proposition. Let p, be a finite measure on F. For any E £ F, there is a sequence 
E n of multi-cylinders (see Terminology B.l) in F with 

lim p(E n AE) = 0, 

n 

where A stands for the symmetric difference. If in addition E £ Ft for some t > 0, then 
the approximating multi-cylinders E n can he taken in Ft. ■ 

As a consequence we see that two finite measures on T> coinciding on the family of 
cylinders, are actually equal. 

We go on, as announced, by performing a converse construction, namely by defining 
through the coupling matrix A, for any a G S, a suitable function on cylinders and then 
uniquely extending it to a probability measure on V. 

For a probability vector a € M m , we define for any cylinder C(t\, ■ ■ ■ , Ji, • • • ,jk ) 

k 

Ma(C(*i,-" ,4; ji,--- ,j k )) = (ae~ tlA ) JJ . (3.1) 

n ' h-i h 

This function enjoys the following key properties: 

(i) it is, for any k € N, a probability measure on the family of cylinders of the 

form C (t \, • • * i,--- , jk) obtained by keeping ((ti, - - - ,tk ) fixed and varying 

(ji>''' ijk) i n {1, • • • ,M} fc , which is actually a cr-algebra being in a one-to-one 
correspondence with the family of all subsets of {1, • • • , M} fc ; 

(ii) if (tj 1 , • • • , t H ) is a subsequence of (t \, • • • , tk) with / < k then for any (J* , • • • , j*) € 

{!>••• j M } 1 

5 5 5 ‘ ' Aij) = ^ ^ A l a(C(^l) ijk)')] 

(jl,-,3k)€J 
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where 

J = {(ji, • • • ,3 k) | ji m = 3i m for m = 1, • • • , /}. 

The latter condition is known as the Kolmogorov Consistency Condition and its validity 
in this context depends upon e -sA being a stochastic matrix for any s , which is in turn 
equivalent, as showed in Proposition A.5, to requiring (H4), (H5) on the coupling matrix 
A. 

We are then in position to use the Kolmogorov Extension Theorem, see for instance [13, 
Theorem 14.36], [20, Theorem 1.2], which ensures, under the previous conditions (i), (ii), 
the existence of an unique probability measure, denoted by P a , on (V,T) which extends 
fi a on the whole T. 

It comes from (3.1) that the map 

a i—>■ P a is linear, 

consequently the measures P a , for a = (ai, • • • , a m ) varying among probability vector of 
M m , make up a simplex of measures spanned by P* := P ei , for i G {1, • • • , M}, and 

M 

p a = y^PD 

i= 1 


Since by (3.1) the measures P* are supported in T>i G Eq (see (B.4) for the definition of 
T>i), we also deduce 

M 

p.A) = £ ai Pj (A D Vi) for any A G E, 

2 — 1 


and 

ai = P a (T>i) for any i G {1, • • • , M}. 

Also notice that all measures P a corresponding to strictly positive a are equivalent in 
the sense that they have the same null sets, and these are the E G T with 

Pj(-E') = 0 for any i. 


3.2. Terminology. By a random variable we mean any measurable map from (V,E) to 
a Polish space endowed with the Borel cr-algebra. A simple random variable is one that 
takes on finitely many values. We denote by E a the expectation operators relative to P a , 
and put for simplicity E; in place of E e; . We say that some property holds almost surely, 
a.s. for short, if it is valid up to a P a -null set, for some, and consequently for all a > 0, 
where > must be understood componentwise. 
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We consider the push-forward of the probability measure P a , for any a € S, through 
the flow cfrh on V defined in (B.8). For a cylinder C := C(ti, ■ ■ ■ ■ • • , jk), we have 

for any a G S 


</>/i#P a (C') = IV{w | e C} = P a (C(ti + h, ■ ■ • ,4 + h;ji, ■ ■ ■ ,jk)) 


k-1 


= ae 




1=2 


n e -(h-h-i)A =P ae -,A(C) 

3 1 , „ v '3l 3l-i 


which implies 


0/ l #P a (£') = P ae _hA(f;) for any E G J. 


We have therefore established: 


3.3. Proposition. For any a € S, h > 0, 

0/i^Pa = Pae-^A- 


Accordingly, for any measurable function / : X> —> R, we have by the change of variable 
formula 

Ea/(0h)= / /(0ft(w))<flP a = f /M# fe #P a = E ae _ A ,/. (3.2) 

JD JD 

We consider, for t > 0, the random variables with values in {1, ■ ■ ■ , M} given by the 
evaluation maps at t, i.e., u i-a u(t). By (3.1), 

w(t)#P a (z) = P a ({cu | w(i) = i}) = (ae" fA ) j 

for any index i € {1, • • • , M}, so that 

w(f)#P a = ae“ tA . (3.3) 

Consequently, if we look at an M-dimensional vector, say b, as a (measurable) function 
from {1, • • • , M} to R, we have 

E a& w (i) = ae _tA • b. (3.4) 


Formula (3.3) can be partially recovered for measures of the type PaLF? (P a restricted 
to E ), where E is any set in F. 

3.4. Lemma. For a given a £ S, E & Ft for some t > 0, we have 

w(s)#(P a LF) = (w(f)#(P a LF)) e -( s ^i A for any s>t. 

Proof: We first assume E to be a cylinder 

A C(ti, ■ ■ ■ , tk, j \, • , jk) 

for some times and indices. Then the condition E £ Ft is equivalent to t > tk- We have 

w(4)#(P a Lfi)(i) =P a (FnC(4;i)) 
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which implies 

w(4)#(P a LE) =P a {E)e jk 
and, according to the definition of P a hi (3-1) 

u;( S )#(P a L£) = (w(4)#(P a LE)) e -( s -*fc) A for a > tk . 

Consequently, 

w(s)#(P a Ll?) = (w(4)#(P a L£)) e -(*-*fc)A e ~(s-t)A = ( w (t)#(p aL £;)) e -( s ~ t ) A 

for s >t, as claimed. The result can be extended by linearity to any multi-cylinder. 

Finally, if E is any set in T, then we consider a sequence of multi-cylinders E n in Tt 
with F a (E n AE) 0. By Proposition 3.1, 

limw(s)#(P a LE n )(i) = limP a (.E n n C(s; i)) = P a (E n C(s; i)) =w(s)#(P a L£)(«). 

n n 

Therefore, 

w(#(P a L£) = hmu(s)f(P a L E n ) = (w(t)#(P a L£)) e"**"*^. □ 

n 

3.2. Stopping times. A stopping time, adapted to Tt, see Appendix B, is a nonnegative 
random variable r, see Terminology 3.2, satisfying 

{r < t} G Tt for any t, 

which also implies {r < f}, {r = t} € J 7 *. 

For a bounded random variable r, we set 

T » = £ [O' - l)/ 2 ”.i/ 2 ”)}), (3.5) 

j 

where I(-) stands for the indicator function of the set at the argument, namely the function 
equal 1 at any element of the set and 0 in the complement. The above sum is finite, being 
r bounded, so the r n are simple stopping times, and letting n go to infinity we get: 

3.5. Proposition. For a bounded stopping time t, r n defined as in (3.5) make up a 
sequence of simple stopping times with 

r n > r, Tn —* r uniformly in V as n —>• oo. 

We consider a simple stopping time of the form 

i 

T = 

3=1 


(3.6) 
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where the sequence t 1 , • • • ,ti is strictly increasing and Ej are mutually disjoint sets of F, 
in addition Ej £ Ft- by the very definition of stopping time. The symbol I(-) stands again 
for the indicator function. 

We define 

Ej = {t> 

so that 

Fj £ F tj _ 1 for any j. 

It is clear that 


j 

Ej = P) Fi \ Fj + i, Fy = V, 

i=1 

3 ~ 1 

F j =V\\jE i for j > 1, F t = E,. (3.7) 

i= 1 


We derive that r can be equivalently expressed as 


i 

t = 


3 = 1 


(3.8) 


where we have set to = 0 to simplify notations. The two expression of r given by (3.6), (3.8) 
are different: in (3.6) the sets Ej are mutually disjoint while in (3.8) they are decreasing 
with respect to j. 


For a stopping time r, we consider the map defined as 

a eA P a , (3.9) 

since the push-forward of P a through uj(t) is a probability measure on {1, • • • , M}, which 
can be identified with an element of S, we see that the relation in (3.9) defines a map from 
S to S which is, in addition, linear. Thanks to Proposition A.2, it can consequently be 
represented by a stochastic matrix, we will denote analogously to the case of deterministic 
times, see (3.3) , by e~ AT , acting on the right. In other terms 

ae~ rA = cu(r)#P a for any a £ S. (3.10) 

3.3. Admissible controls. We call control any random variable H taking values in 27(0, +oo; P A ) 
such that 

(i) is locally (in time) bounded, i.e. for any t > 0 there is R > 0 with 


sup |E(i)| < R 

[o A 


a.s. 


(3.11) 
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(ii) is nonanticipating , namely for any t > 0 

oj\ = W 2 in [0,f] =4> H(a;i) = E(u> 2 ) in [0,t]. (3-12) 

Second condition can be equivalently rephrased requiring S to be adapted to the filtra¬ 
tion J~f . namely requiring that S (t) is Z)-measurable for any t. In fact, if (3.12) holds 
true then the value of E (u>)(t) just depends on the restriction of u to [ 0 , t] which actually 
implies that E (t) is Z)-measurable. The converse implication comes from a version of 
Doob-Dynkins Lemma for Polish spaces, see [12] Lemma 1.13, asserting that if the a- 
algebra spanned by a random variable # 1 is contained in that spanned by # 2 then 7 ^ 1 
is a measurable function of # 2 . In our case # 1 is S(s) for s £ [0, t] and # 2 is 

uj i-A restriction of w to [0, t] 

which takes value in D(0, t; {1, • • • , M}). 

Being the paths in Z>(0, + 00 ; l w ) right continuous, the condition of being adapted 
implies, see [20] p. 71, that E is in addition progressively measurable, namely, for any t 
the map 

(s, u>) e -7 5(s, u>) 

from [ 0 , t] x T>(0, + 00 ; { 1 , • • • , M}) to l w is measurable with respect to the cr-algebras 

£>[0, f] x Ft and B, where £>[0,t], B denote the family of Borel sets of [0, t] and M. N with 
respect to the natural topology. We will denote by /C the class of admissible controls. 

For a control E, Z(S) is also a random variable with values in C (0, + 00 ; T^), in addition 
Z(E) is adapted and consequently progressively measurable. 


For a time t, we say that a control is piecewise constant in [0, t] if it is of the form 

m 

^2x k I([s k ,Sk + i)) in [ 0 , t) 
k =1 


for some F Sk -measurable M^-valued bounded random variables X k , where 

s k is an increasing finite sequence with si = 0, s rn = t (3.13) 

and I(-) is as usual the indicator function. For any control E and s k as in (3.13), then the 
E (sk) are F Sk -measurable M^-valued bounded random variables for any k, so that 


f YlkLi^is^Kisk^k+i)) in [ 0 , t.) 

\ E in [; t , + 00 ) 


is a control piecewise constant in [0 ,t]. We therefore directly derive from Proposition B.3: 
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3.6. Proposition. For any control E andt > 0, there is a sequence of controls E n piecewise 
constant in [0, t] and locally (in time) uniformly bounded with 

E n —> E in the Skorohod sense in T>(0, +oo;M iV ), for any oj. 


4. An ESTIMATE FOR SUBSOLUTIONS 


For a > 7 , an initial point x in T N , a bounded stopping time r and a control 
consider in this section the action functional 


we 


E a 


/ L u ^(x+l{E)(s),-E(s)) + ads 

Jo 


(4.1) 


Notice that /(H)(r) belongs to T A for any u, see (B.9). The meaning of the sum between 
elements of T N is made precise in Notation 2.1. 

We aim at proving: 

4.1. Theorem. For a > 7 , let u, r, E, a be a subsolution to (HJo), a bounded stopping 
time, a control and a probability vector in S, respectively. For any initial point x £ T N , 
we have 


EaK,(o)(» - Uu:( T )(x + I(H)(r))] < E a ( L u ^(x + 1(E)(s), -S(s)) + ads 

Uo 


(4.2) 


The difficulty in proving Theorem 4.1 is that the two integrals appearing in (4.2) do not 
commute due to the presence of the random time r. It is worthwhile to point out that this 
difficulty never happen in the study of evolutionary problem for weakly coupled systems 
(see [17, Proposition 2.5] for more details). Joint measurability properties guarantee that 
the Fubini theorem can be applied in regions where stopping time is constant. The idea 
is then to approximate r by a sequence of simple stopping times r n and then exploit the 
subsolution property of u separately in the regions where r n are constant. We will take 
advantage of some properties about probability measures P a we have gathered in Section 
3. 

Throughout the section we put a = 0 to ease notations. 

4.2. Lemma. Let u, a be as in the statement of Theorem 4.1, we further consider t^ > 
t\ > 0, E £ , £0 6 £>(0, + 00 ; W N ), and zq £ T w . Then 

/ (/Mtp^o) - u u (t 2 )(zo +Z(to)(h - tl))) cflPa 
Je 


< 


J ^ E u ,( s )(zo+T(^o)(s-ti),-?o(s))^ cflPa 
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Proof: We can assume zq = 0 without loosing generality in the proof. Since u is a 
subsolution to (HJa), we have 

-p-q< Li(z, -q ) + Hi(z,p ) < Li(z, -q) - A* u(z) (4.3) 

for any i G {1, • ■ ■ , M}, z G T N , q G R N , p G dui(z) (see Remark 2.4). We define 

d = W (i 1 )#(P a L£), 

and we have for a.e. s G (£ 1 ,^ 2 ) 

^ (( de ~ ( '” <l)A ) ' U ( X (&)( S - *i))) 

= ^de _(s_fl)A j • ( -Au(I(( 0 )(s - ii)) + (p 1 (s - ti) • Co (s — ti), - - - ,p M (s - h) • Co (s ~h 

where p l (s — t\) is a suitable element in dui(I(£ o))(s — t \) for any i. Combining this last 
equality with (4.3) and setting L = {L\, ■ ■ ■ ,Lm), we deduce 

-4 ((d e -(-J A ) • u{I({„)(„))) < -L(Z(&)(»),-&(»)), 

and consequently 

d • u(J(Co)(fi)) - d • e _(<2_<l)A u(X(Co)(t 2 )) = £ ((de“ (s_tl)A ) • u(J(Co)(a))) ds 

< J t h (de- Ml)A ) • (L(X(Co)(s), —Co(s)) ds. 

We have by the definition of d, (3.4), Lemma 3.4, E G Et x 

[ ( Mtl) (X(Co)(ti))-^ 2) (X(Co)(t2)))dPa = d-(u(X(Co)(ti)-e-( t2 - tl ) A u(X(Co)(t 2 )) 
Je 

f L w(s) (X(Co)( S ),-Co(s)) = (de“^ _tl ^ A ) • (L(X(Co)(s), —Co(s)) 

•J E 

for any s in [ti , ^ 2 ] - By plugging these relations in the last inequality and using the Fubini 
theorem, we get 

/ (iMq)(X(Co)(t!)) - u w(t2) (X(Co)(t 2 ))) dPa < [ ( f 2 (LMi 0 ), -Co) ds) dP a . □ 

Je Je \Jt 1 / 

4.3. Lemma. For a control S and a bounded stopping time r, let E n , r n be sequences of 
controls and bounded stopping times, respectively, with 

E n —> E a.s. with respect to Skorohod metric (4-4) 

uniformly in V (4.5) 

a.s. for any n. 


T n ^ T 
T n > T 
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Assume in addition that for any T > 0, there is R = R(T) > 0 with 

sup |H re (s)| < R a.s. for any n 
se[o,T] 


converges in M to 


d-'u T -^(^n )> l —‘n) ds 


L UJ (x + l(E), -E) ds 


Ea[M w (o)(^) ~u u ,( Tn )(x + l(Z n )(T n )\ -» E a [iL,(o)O0 - u u{j) {x + X{E){r)\ (4.7) 

for any x € M. N , a € S. 

Proof: We set x = 0. We know that conditions (4.4) (4.6) hold true outside a P a -null 
set denoted by V. If u G T> \ V the H n (cu) are uniformly bounded in [0, r(cu)], and we 
derive from (B.l), (B.5) that E n (uj) converges pointwise a.e. in [0,r(cu)] to H(cu). Taking 
also into account the continuity of L and X(-), see Proposition B.7, we get through the 
dominated convergence theorem 


I T LJ (X(s r) ), s n ) ds 
Jo 

Let T be such that r < T a.s., by (4.6) 


L W (Z(H), —H) ds a.s. 


max |X(H n )(s)| < RT for any n, outside a P a -null set, 
se[o,T] 


and consequently the sequence 


I d-'ui (-^(^n) > —-n) ds 

Jo 


is a.s. uniformly bounded. Taking also (4.8) into account, we can thus obtain the claimed 
convergence with t in place of r n in the approximating sequence, again via the dominated 
convergence theorem. We further have 

[ L u (l(E n ),-E n )ds - [ L u (l(E n ),-E n )ds < f \L u (l(E n ),-E n )\ds 

Jo Jo Jt 

Owing to (4.5) and the uniformly boundedness property of the integrand, the right hand- 
side of the above formula becomes infinitesimal, as n goes to infinity, uniformly in oj so 
that 

E a f L u (I(z, n ), — s n ) ds — f L u (I(^ n ), — s n ) ds —> 0. 

.Jo Jo 
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This shows the first convergence in the statement. Limit relation (4.7) can be proved 
similarly using the continuity of u in T N . □ 


4.4. Lemma. Assume 


i 

t = 

3 = 1 


(4.9) 


to be a simple stopping time, with the tj making up an increasing sequence of times, and 
set Fj = {t > tj} for any j £ {1, • • • , l}. Let u, E, a, x be as in the statement of Theorem 
4-1, then 


E a 


L ul (x + 1(E), -E) + ads 


E 


j=1 J F j 


L u (x + 1(E), -E) + ads dP a 


Ea [« w (0) i x ) ~ u cj(t) (X + T(E(r) 

l 

= E 


j.i ■>”, 


(“wfo-ijOc + z ( H ( t i-i))) “ u oj(tj)(x + l('B(t j )))) dP a 


Proof: We set to = 0 and 

I = E a 


L u) (x + 1(E), -E) + ads 


'o 


Taking into account the definition of r and that the tj are monotone, we have 

/ /» \ % n • 

/ = E / / L u (x + 1(E),-E) + ads = ^2^2 / / L u (x + 1(E),—E) + ads 

i=\ ^Fi JO i=l j=1 ^Fi Jtj —l 


= EE / / E a ,(x+X(E),-E) + a ( i.s 

j=l i>j ^ •'b-i 

and, owing to (3.7) 


E / / L w (a:+ I(S), —H) + ccds = / / L^(x + Z'(E), —E) + a ds 

for any j € {1, • • • , ^}. Therefore, summing over j we get 


' = E 


j=\ J Fj \ Jtj -1 


L^(x + Z(S), -E) + ads dP a 



A LAGRANGIAN APPROACH TO WEAKLY COUPLED HJ SYSTEMS 


17 


as desired. The second equality in the statement can be proved along the same lines, we 
provide some detail for readers’ convenience. We start defining 

J = E a [u w(0) (x) - it w ( T )X + I(S(r)))], 

then we have 

J = 22 f (“w(0)( a: ) -U «(tj)( S+2: ( S ( t j)))) dIP a 

i =1 jEi 

l i r. 

= 2222 (^(t i _ 1 )(* + ^(S(ij-i)-« w( t J .)(® + 2:(H(tj))))<® , a 

i= 1 j =1 

= 2222 [ (Mtj- 1 )(* +x ( 3 ( i J-i) - ' u «(tj)( s+2: ( s ( t j)))) dIp a 

J=1 i>3 JEi 

and, again exploiting (3.7) 

*>J JE ‘ 

= [ («w(t i _ 1 )(* + 2:(S(ij-i)-« a ,( tj )(x + X(S(tj))))<® , a 

J Fj 

for any j € {1, • • • , i}. We conclude the proof summing over j. 

□ 


4.5. Proposition. The assertion of Theorem 4.1 is true if we take the stopping time r 
simple, say of the form (4.9), and the control S piecewise constant in [0, T] for some 
T>ti. 

Proof: Since T >ti, we can assume that E has the form 

m 

E = y]X fe I([sfe-i,Sfe)) in [0 ,ti), 
k=1 

where X*. are M^-valued random variables and s*. is a finite increasing sequence with 
so = 0 and s m = tf, we can assume in addition that all the times tj, j = 1, ■ ■ ■ ,1 belong to 
the sequence. Consequently, it can be univocally associated to any interval [sfc_i,Sfc) an 
index j such that [s*,_i,Sfc) C [tj-i,tj). Due to the nonanticipating character of E 


Xfc is J 7 Sfe _ 1 -measurable. 


(4.10) 
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We fix indices k, j such that [s k -i,s k ) is contained in [tj-i,tj), by (4.10) there is a 
sequence of simple J r Sfc _ 1 - random variables 

Y n = J2Vr W) 

r 

taking values in R N and converging a.s. to X k , see [14, Theorem 1.4.4], with y” G and 
B'j G F Sk _ 1 for any n. Then, slightly modifying the argument in Lemma 4.3, we get that 


Fj Js k _i 


L u (s)(Z(z.( s k- 1 )) +I(Y n )(s — Sfe_i), — Y n ) ds 


converges to 


J Fj 

as n goes to infinity, and similarly 


rsk 

/ L u ^{Z(z>(sk- 1 )) + I(X k )(s — Sk- 1 ), —X k ) ds 

j SL_1 


converges to 


(^w(sfc_i) l)) ^uj(s k ) l) T Y(Xk){sk Sfc—l))))) 


Due to the form of Y k , the integral in (4.11), (4.12) can be in turn written as 


FjCBr 4 Sfc „ 1 


L u (s)(Y{E(s k - 1 )) + y™ (s - Sfe_i), -y”) dsdP a , 


r 3 ' n 


(4.11) 


{u^sk^m^isk-i)) - Uuj(s k )(Y(E(s k _ i) + X(Y n ){s k - s fe _i))))) (4.12) 


{ _i) (-^(“("Sfc—1)) ' l ^u}(s k ) l) T y r {s k S k —\)))) } dP a 


respectively. Since Fj € J 7 t j _ 1 , G J r Sfc _i and s^-i > ij-i, we deduce Fj n G J r Sfc _ 1 , 
and we can apply Lemma 4.2 to any term of the previous sum. This yields 

/ (««(,*_!)(Z(3(s fc -i)) - n w(s)c) (X(“(.s fc _i) + y” (s fc - s fc _i))))) cflP a 

<[ f -C' W (s)( 2: (3(sfe-i))+y"(s-s fc _i),-y")dscflP a 

JFjnBr Js fc „ 1 

for any r. By summing over r and passing to the limit as n goes to infinity, we further get 
/ Kfsnjt^tsfc-i)) - ^( Sfe )(^(3(s fc _i) +X(X fc )(s fc - s fc _i))))) dP a 

< [ f L^ s )(I(E(s k -i)) +F{X k ^i)(s — s k -i),—X k )dsdF a . 

J Fj J Sk —i 



A LAGRANGIAN APPROACH TO WEAKLY COUPLED HJ SYSTEMS 


19 


By summing all inequalities as above corresponding to intervals [sk~i,Sk) in [tj-i,tj) we 
obtain 


(Z(H(tj))))) dP a < 


Pj J Fj Jtj —i 

We conclude the proof summing over j and exploiting Lemma 4.4. 


L u{s){Z{z.{s)), - s 0)) ds ^a- 

□ 


Proof of the Theorem 4.1. By Proposition 3.5 r can be approximated uniformly in oj 
by a sequence of simple stopping times r n with r n > r and r n < T for some constant 
T, in addition by Proposition 3.6 E can be approximated a.s. with respect to Skorohod 
metric by a sequence of control E n piecewise constant in [0, T] and and locally (in time) 
uniformly bounded. 

Owing to Proposition 4.5, inequality (4.2) holds true if we replace r, E by r n , E n , 
respectively, for any n. We conclude by passing at the limit as n goes to infinity and 
exploiting Lemma 4.3. □ 


4.6. Notation. For a bounded stopping time r and a pair x, y of elements of T N , we set 
/C(t, y — x) = {S <e K | Z(E)(r) = y-x a.s.} , 

notice that both 1 (E)(t) and y — x are elements of T iV , see (B.9) and refer to Notation 2.1 
for the meaning of y — x. Also notice that I(E)(t) is a random variable taking value in 
because E is progressively measurable and r is a stopping time. We recall that the diction 
a.s. must be understood with respect to the family of equivalent measures P a , a > 0. 

We will call, with some abuse of language, the controls E belonging to JC(t, 0) r-cycles. 


4.7. Remark. For x, y in T N , the family of controls /C(t, y — x) is nonempty whenever 
ess inf r > 0. In fact for such a stopping time select e > 0 with e < ess inf r and define a 
control 5 setting for any u 


E(u)(s) 


zo for s € [0, e) 

0 for s £ [e, +oo) 


where zo is any vector of with proj(e^o) = y — x (proj is the projection of onto 
T A ). It is indeed apparent that E belongs to /C(r, y — x). 


Using Notation 4.6, we derive from Theorem 4.1: 

4.8. Corollary. For any pair of points x, y in M. N , subsolution u to (HJa), a € S, bounded 
stopping time t and 5 £ /C(r, y — x), we have 

L uj(s)( x + + ads 



E a[« w (0)(z) - ««(t)(J/)] < E a 


(4.13) 
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In the next section we will show, see Theorem 5.7, that (4.2) actually characterizes 
subsolutions to (HJa). 


5. A REPRESENTATION FORMULA FOR SUBSOLUTIONS 


Throughout the section we consider a constant a greater than or equal to 7 . For y in 
R N , b € M M , we define 


Vi(x) = inf Ej 


Luj({x + T(rP), 


Uo 


+ OL ds + 


(5.1) 


for any i £ {1, • • • , M}, x £ , where the infimum is taken with respect to any bounded 

stopping times r and E £ /C(r, y — x). We have 


5.1. Proposition. The function v = (iq, • • • ,vm) defined in (5.1) is bounded in T N . 


Proof: Taking into account that 1 € ker(A), we see that if bo £ F a (y) (see (2.1) for the 
definition of F a ) then bo + //1 £ F a (y) as well, for any // £ M. We can consequently find 
a subsolution u to (HJa) with 

u(y) < b. 

Owing to Corollary 4.8, we then have 


E,- 


/ F u ; ( s )(x + T(S)(s), —S(s)) + ads + 

Jo 


> Ei 



U 0 

for any i £ {1, • • • , M}, x £ R N , 
implies 


L w ( 8 ) ( x + Z ( H ) ( s ), -S(s)) + a ds + u u ( T) (y) 
bounded stopping time r and 5 G K,(r,y 


> Ui(x). 

— x). This 


v(x) > u(x) for any x, 

where > must be understood componentwise. On the other side, by setting r = \x — y\, 
5 = ZZ_y\ an( l taking into account that L is locally bounded, we see that v is also bounded 
from above. □ 


We aim at showing: 

5.2. Theorem. The function v defined by (5.1) is subsolution to (HJa). 

We postpone the proof after some preliminary material. The crucial point is to prove a 
Dynamical Programming Principle type result. We will use the flow (ph defined (B. 8 ) in 
Appendix and the change of variable formula (3.2). 
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5.3. Proposition. Let h, x, £o? j be a positive time, a point in R N , a path inV(0, +oo; M w ), 
and an index in {1, • • • ,M} respectively. Then 

f*h 


Vj(x) < Ej 


f L w (x + l(£o),-£o) + ads + v u r h \(x + l(€o)(h)) 
Jo 


(5.2) 


Proof: Fix e > 0 and set a = 0, z = x +Z(£o )(h) to ease notation. Denote, for any i, by 
t 1 , Sj bounded stopping times and controls in tC(r l ,y — z) with 


Vi(z) > E,; 


/ L u (z + —Hi) ds + b u r T %\ 

Jo 


— £ 


(5.3) 


We define new stopping times and controls via 

r = t 1 , E = Ej in T), for any i, 
it is clear that E £ JC(r, y — z). We set 

t(lo) = T{4>h{u)) + h for any ui £ V, 
this is yet a stopping time, since for any t > h 

{uj | t(u) <t} = {uj | r((j) h (uj)) <t - h} = | r(u) < t - h }), 

which actually yields by Proposition B.5 

{w | t(w) <t} £ 7 U 

as desired. We further set 


E(w)(s) = 


_ / £o(s), 


for uj £ V, s £ [0, h) 


E(0/ l (cu))(s — h), for oj £ T>, s £ [h, +oo). 


To justify E being an admissible control, we define a map T from D(0, +oo; M w ) to itself 
through 

= / &(s) for s £ [0, h) 


£(s — h) for s £ [h, +oo). 

According to the very definition of convergence in D(0, + 00 ;]^), this mapping is contin¬ 
uous in the sense of Skorohod, in fact if £ n —>• £ and g n is the corresponding time scale 
deformation, then we define 

, , _ f s for s € [0,h) 

y n ^ ' \ g n (s — h) + h for s £ [h, + 00 ) 

and it is straightforward to check that g n locally uniformly converges to the identity 
function in [0,+ 00 ) and V^(£)(< 7 n (s)) locally uniformly converges to ^(^)(s). We can 
rephrase the definition of E above as 

E (w) = T(E (</> h (u)), 
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which shows that S is a random variable as composition of continuous and measurable 
maps. If u\ = W 2 in [0 ,t], for some t > h, then 


which implies 


therefore 


1 ) = (j>h(u 2 ) in [ 0 ,t- h] 


E(4> h (oJi)) = E(4> h (oj 2 )) in [ 0 , t — h], 


H(wi) = Co = S(w 2 ) in [0, h], 

E(ui(s)) = E(tp h (ui))(s - h ) = E(4> h (u 2 ))(s - h) = E(cj 2 )(s) in [h,t\, 

which shows that S is nonanticipating. Finally the the uniformly boundedness condition 
is clearly fulfilled. We conclude that E is an admissible control. To show that it belongs 
to /C(r, y — x), we consider for u G V 

/ E(u)ds= / Co ds T / E(<t> h (u))(s - h) ds = z - x + / E(<j> h (u))(s) ds, 

Jo Jo Jh Jo 

Owing to E G /C(t, y — z) and Proposition 3.3 we have for any a > 0 in S 


rr(<t>h(u)) | j r T (u) | 

u\ / E((j) h (u;))(s)ds ^ y - z} =¥ ae -hA <u \ / E(u)(s) ds ^ y - z> = 0. 


/o 


1 0 


This establishes that E e /C(r,y — x). We compute for s > 0: 

~ r h ps-\-h ~ 

x + l(E)(u)(s + h) = x+ Co dr+ E(u) dr 

Jo Jh 


(5.4) 


= z+ ( E(<j> h (u))dr = z + l(cj) h (u>))(s). 

Jo 


According to the very definition of v, we then have 


Vj (x) < Ej 


= Ej 


L UJ (x + 1(E), -E) ds + b t 


0(r) 


(5.5) 


/ Lu{x + T(Co), — Co) ds + / Lu{x+ T(zJ),—zJ) ds+ b L0 (j) 
1 0 Jh 
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Using the definitions of r, E, the change of variable formula (3.2) and (5.4), we have 


Ej 


=Ej 


=Ej 


=E C 


L u (x + X(s), —E) ds + 6 tJ (f( w )) 


r—h 


Luj( s +h){ x + ^(‘=‘)( w )( s + h), —r,(u)(s + h)) ds + bu>(j(w)) 


r T (<t>h(.u)) 


L 4> h {w )(* + Z(E)(0 fc (w)(s), -5 (<j> h {u){s)) ds + 6^( w )(r(0h(w))) 


r r M 


L w (z + Z(E)(w)(s), -S(w)(s)) ds + b u(T) 


Using (5.3), we further get 

c t(u) 


E p , „-h 


r T (“) 

/ L w {z + X(S)(w)(s), -E(w)(s)) ds + 6 w(r) 
Jo 


E( 


e,e 


-ft A 




Xu;(£ + X(^j)(s), ~ =L i( s )) ds + 6 w ( T i) 


< 


^2 ( e i e hA ' e ») (^( z ) + e ) = e j e hA ■ V (U> + £ = K j v cu(h)(z) + £• 


Combining the last two computations we get 


Ej 


L w (a: +X(s), — r,) ds + 6 w (f( w )) 


< ^j v oj(h)(z) 


and recalling (5.5) and the definition of 2 we finally obtain 


'Wj(.'c) < Ej 


U 0 


L u (x + Z(£ 0 ), ~6)ds + ^(ft)(^ + X(^ 0 )(h)) 


+ £. 


Taking into account that e is arbitrary and that we have set a = 0, we obtain in the end 
the assertion. □ 


5.4. Lemma. The function v defined by (5.1) is Lipschitz-continuous in T N . 

Proof: We consider two points z x, and set to = \z — ar|, Eo = ]fEf| =: Q- Then, 
according to (5.2) 


vfix) - e,; e z|A • v(z) < Ej 


r\ x ~ z \ 

/ L u r s )(x + sq, -q) + ads 
Jo 
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from which we derive 


< E . 

/ ^( S )(s + s9, -?) +ads 

V / 

Jo 


(5.6) 

We take a constant R which is at the same time upper bound of both L(x, q) in T iV x f? (0,1) 
and |v(x)| in T N , see Lemma 5.1, and in addition Lipschitz constant of 

t eA e* e~ tA in [0, +(xj) 

for any i, see Proposition A. 6 . We deduce from (5.6) 

Vi(x) — Vi(z ) < (R + a + R 2 ) \x — z\. 

This completes the proof. □ 


Proof of Theorem 5.2. We consider a point x € where all components of v(x) 

are differentiable, and fix a nonvanishing vector q G , further we take £o = q, and 
accordingly 

x + X(£ 0 )(s) = x + s q for any s > 0. 

Formula (5.2) then reads 

f h 

Vj(x) — Gj e~ hA ■ v(x + hq) < / e 3 e~ sA ■ L(x + s q, —q) + a ds , 

Jo 

which implies 

vj(x) — e~ e~ hA ■ v(x + hq) 1 rh 


h 


<— I e 7 - e sA ■ h(x + s q, — q) + a ds. 
h Jo 


Passing to the limit as h goes to 0, and taking into account that all the Vj are differentiable 
at x, we get 

A J • v(x) — Dvj(x ) • q < Lj(x , —q) + a. 

Being q arbitrary, we further obtain 

A 3 ■ v(x) + Hj(x , Dvj(x )) = A 3 v(x) + sup{— Dvj(x) ■ q — Lj(x , — q)} < a. 

This shows that v(x) is a.e. and so viscosity subsolution of the system (HJa). □ 

5.5. Theorem. For y € T n , b € F a ( y) if and only if 


E,- 


VJo 


L U{S ) ('V + (s), -E(s)) + ads-bi + 6 w(t) 


> 0 


(5.7) 


for any *£{!,••■ , M}, bounded stopping times r and r cycles 
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Proof: We denote as usual by v the function defined in (5.1). By taking the stopping 
time r = 0 and the control E = 0 . we see that 

v(y) < b, 

where < must be understood componentwise. If (5.7) holds then we also get the converse 
inequality so that v(y) = b, which proves b G F a (y) being v subsolution to (HJa). 

Conversely, if there is a subsolution u of (HJa) with u(y) = b then (5.7) is a direct 
consequence of Corollary 4.8 □ 


We give a characterization of the Aubry set from the Lagrangian point of view. 

5.6. Theorem. Assume the element b appearing in (5.1) to be in F a (y), then 

(i) v(y) = b; 

(ii) v is the maximal subsolution to (HJa) taking the value b at y; 

(iii) If a = 7 and y € A then v is a critical solution. 


Proof: Item (i) has been already proved in Theorem 5.5. If u is a subsolution to (HJa) 
with u(y) = b, then by Corollary 4.8 we get 


Ui(y ) < Ej 


Uo 


L u (s)(x + X(E)(s), 


—r,(s)) + ads + b u ( T j 


for any i G {!,••■ ,M}, bounded stopping time r and r-cycle £. This shows 


v > u. 


Item (iii) directly comes from the definition of the Aubry set. 


□ 


We finish the section by showing that for any a > 7 inequality (4.13) actually charac¬ 
terizes subsolutions to (HJa). 

5.7. Theorem. A function u : T n — > M A/ is a subsolution to (HJa) if and only if inequality 
(4.13) holds true for any pair of points x, y in T iV ; a G S, any bounded stopping time t, 
£ G /C(r, y - x). 

In view of Corollary 4.8 , it is enough to show: 

5.8. Proposition. If a function u : T N satisfies inequality (4.13) for any pair of 

points x, y in T iV , a G S, any bounded stopping time t, 5 G JC(t, y — x), then u is a 
subsolution to (HJa). 
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Proof: By using the same argument of Lemma 5.4, we see that u is Lipschitz-continuous. 
Fix i £ {1, • • • , M} and take a differentiability point y of U{. define v as in (5.1) with u (y) 
in place of b, then, owing to Theorem 5.6 

v > u in T n and v(y) = u(y). 

Hence Ui is subtangent to Vi at y, which implies Dui{y) £ dvi(y) and, being v subsolution 
to (HJa), by Theorem 5.2 and Remark 2.4 we get 

Hi(y,Dui(y )) + A* u (y) = Hi(y,Dui(y)) + A*v(y) < a. 

This concludes the proof. □ 


Appendix A. Stochastic matrices 

In this appendix we collect some basic material on stochastic matrices. All matrices 
appearing below are square matrices. We refer to [15, 19] for the results stated without 
proof. 

We denote by S C R M the simplex of probability vectors of M m , namely with nonnegative 
components summing to 1. 

A.l. Definition. A (right) stochastic matrix is a matrix possessing nonnegative entries 
and with each row summing to 1. 

A.2. Proposition. A matrix B is stochastic if and only 

a B£S whenever a £ S. (A.l) 

Proof: B is stochastic if and only if all its rows are probability vectors, or, in other terms, 
if and only if 

ej • B £ S for any i. 

this is in turn equivalent to (A.l). □ 

By the Perron-Frobenius theorem for nonnegative matrices, we have 

A.3. Proposition. Let B be a stochastic matrix, then its maximal eigenvalue is 1 and 
there is a corresponding left eigenvector in S. 

By the Perron-Frobenius theorem for positive matrices, we have 

A.4. Proposition. Let B be a positive stochastic matrix, then its maximal eigenvalue is 
1 and is simple. In addition, the unique corresponding left eigenvector belonging to S is 
positive. 
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Even if it is an elementary fact, we give for completeness the proof of the key prop¬ 
erty that the coupling matrix of the Hamilton-Jacobi system under investigation spans a 
semigroup of stochastic matrices. 

A.5. Proposition. For a matrix A, e~ tA is stochastic for any t, if and only */(H4), (H5) 
hold with A in place of A. 


Proof: Assume that A satisfies (H4), (H5), then, given t > 0, I — is stochastic for 
n suitably large, consequently (/— ^- j" is stochastic because the product of stochastic 
matrices is still stochastic, and 


e 


-tA 


lim 

n—>oo 



n 


is stochastic because stochastic matrices make up a compact subset in the space of square 
matrices. Conversely, if e~ tA is stochastic then the relation 


A = lim 

t —>-0 


I - e~ tA 
t 


implies that A satisfies (H4), (H5). 


□ 


A. 6 . Proposition. The function 

t i-A e.j e~ tA 

is Lipschitz continuous in [0, +oo) for any i € {1, • • • , M}. 


Proof: We have 



e 


-tA 


—e,; A e 


-tA 


which is bounded in t € [0,+oo) because the matrices e <A , being stochastic, vary in a 
compact subset of the space of M x M matrices. □ 


Appendix B. Path spaces 

We refer to [2] for more details in this section. The term cadlag corresponds to the French 
acronym continu a droite limite a gauche , namely continuous on the right and with left 
limit. We consider the space of cadlag paths defined in [0, +oo), with value in {1, • • • , M} 
and R N , denoted by D := T>(0, +oo; {1, • • • , M}) and T>(0, +oo; , respectively. For 
any t > 0, we also indicate by D(0, t; {1, • • • , M}) the space of cadlag paths defined in [0, t] 
with values in {1, • • • , M}. It can be proved that 

Any cadlag path has at most countably many discontinuities. 

Any cadlag path is locally (in time) bounded. 


(B.l) 

(B.2) 
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B.l. Terminology. To any finite increasing sequence of times t±, ■ ■ ■ ,tk, with k G N, and 
indices j 1 , • • • ,jk in {1, ■ ■ • , M} we associate with a (thin) cylinder defined as 

,4;iiw ,jk ) = W I uih) = Ji, — ,w(4) = jfc} C T>. (B.3) 

To ease notations, we set 

T>i = C(0; i) for any i G {1, • • • , M}. (B.4) 

We call multi-cylinders the sets made up by finite unions of mutually disjoint cylinders. 

We endow V with the cr-algebra T spanned by cylinders, those of the type C(s;j ) for 
a > 0, j G {1, • • • , M} are indeed enough. A natural related filtration J- t is obtained by 
picking, as generating sets, just the cylinders C(t i, • • • ,tk',ji, • • • ,jk) with < t, for any 
fixed t > 0. 

Same construction, mutatis mutandis can be performed in 2>(0, +oo; , in this case 
the cr-algebra is spanned by cylinders of the type 

{£ G V(0,+oo-,R n ) | £(s) G E} 

for s, E varying in [0, +oo) and in the Borel cr-algebra related to the natural topology of 
M iV , respectively. 

Both T> and X>(0, +oo; M w ) can be endowed with a metric, named after Skorohod, which 
make them Polish spaces, namely complete and separable, and such that the aforemen¬ 
tioned cr-algebras are the corresponding Borel cr-algebras 

B.2. Remark. A consequence of the previous definitions is that T is the minimal cr- 
algebra for which the evaluation maps 

t !->■ oj(t) t G [0, +oo) 

are measurable and the same holds true for the cr-algebra in D(0,+oo; M^) with respect 
to the evaluation maps 

£ ^ £(*)■ 

A map E : T> —> T>(0, +oo; M^) (resp (j) : D —> V) is accordingly measurable if and only 
if the maps u i-A S(c u)(t) from V to (resp., u i-A ^>(w)(t) from V to {1, • • • , M}) are 
measurable for any t. 

The convergence induced by Skorohod metric can be defined, say in T>(0, +oo;R^) to 
fix ideas, requiring that there exists a sequence g n of of increasing continuous functions 
from [0, +oo) onto itself (then g n ( 0) = 0 for any n) such that 

g n {s) —> s uniformly in [0, +oo) 

€n(9n(s)) €(s) locally uniformly in [0,+oo). 
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This is basically locally uniform convergence, up to an uniformly small deformation of 
the time scale given by the g n . We infer from the previous definition that 

£ n —> £ in the Skorohod sense => £ n (i) —>• £(t) at any continuity point of £ (B.5) 

which in particular implies 

£ n -» £ in the Skorohod sense =$■ £ n (0) —> £(0) (B.6) 

We moreover have 

Any sequence convergent in the Skorohod sense is locally uniformly bounded. (B.7) 


For t > 0, we say that a path in D(0, + 00 ;!^) 
form 


is piecewise constant in [0, t] if is of the 


1 -1 

x k I([sfc, Sfc+i)) for s € [0, t) 

k =1 


where Xk € and Sk is an increasing sequence of times with s± = 0, si = t. We will 
use the following approximation result, see [2] Section 12, Lemma 3, in a version, slightly 
accommodated to our needs. 


B.3. Proposition. For t > 0 and £ € P(0, + 00 ; R A ), let sk = 1, ■ ■ ■ , l n , be a family of 
strictly increasing finite sequences with s” = 0, sf = t and 

sup sf — —> 0 as n goes to infinity 

k 


then the sequence of (piecewise constant in [0, t\ ) paths 


\ £ in [t, + 00 ) 


converges to £ in T>(0, + 00 ; M w ). 


For any h > 0, we consider the shift flow (j)h on T> defined by 

4>h(cu)(s) = uj(s + h ) for any s € [0, + 00 ), u G V. (B.8) 

Notice that cfh is not in general continuous since the fact that u n —> ui in the Skorohod 

metric does not in general implies that 0fe(w n )(O) = co n (h) —> (f)h(u)( 0) = u(h), unless of 

course h is a continuity point for u, and so does not in turn implies, by (B.6), that 4>h(L0 n ) 
converges to However we directly derive from Remark B.2: 

B.4. Proposition. The shift flow <fh '• V is measurable for any h > 0. 
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B.5. Proposition. For nonnegative constants h, t, we have 

C F t +h- 

Proof: For any t\ > 0, j\ £ {1, • • • , M} we have 

= C(h + h,ji). 

The assertion thus comes from the fact that Ft is spanned by cylinders of the form C(t\\ ji), 
with t\ < t, and in this case C(t\ + h;j i) £ Ft+h- □ 

We also consider that space C (0, +oo; T iV ) of continuous paths defined in [0, +oo) taking 

values in T N . It is endowed with a metric giving it the structure of a Polish space, which 
induces the local uniform convergence. 

We define a map 

X :V(0, +oo; R N ) ->• C( 0, +oo; T N ) 
via 

£(£)(£) = P r °j (J fds^j . (B.9) 

where proj indicates the projection from onto T N . 

B.6. Proposition. The map T( ) is continuous. 

Proof: Let us consider a sequence £ n in ^(Oj+oosM^) converging to some £, then by 
(B.7) it is locally (in time) uniformly bounded and by (B.l), (B.5) 

£n(s) ->• ^(s) a.e. in [0,+oo). 

Then by the dominated convergence theorem and continuity of proj 

^(£n)(*) for any t. (B.10) 

Furthermore, from the uniformly boundedness of and the fact that proj is nonexpan- 
sive, we derive that the Z(£ n ) are locally equiLipschitz continuous and locally uniformly 
bounded. By the Arzela-Ascoli theorem with (B.10), we get 

I(Cn) T(^) locally uniformly in time, 

as desired □ 

For uj € V, t > 0, x € R N , we consider the function 

£>->•/ L u(s){x+ I(0( s ),~t( s )) ds (B- 11 ) 

Jo 

from P(0, +oo;l w ) to R. 

B.7. Proposition. The function defined in (B.ll) is continuous. 
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Proof: Let £ n be a sequence converging to some £ in X>(0, +oo; R^), then the £ n are 
uniformly bounded in [0,f] and converge pointwise to £ a.e by (B.l), (B.5), (B.7). Fur¬ 
thermore, bearing in mind Proposition B.6, we know that Z(£ n ) converges to Z(£) in 
C(0,+oo; T A ). Using the continuity of L ,, for any i, we derive that 

L u(s){x + X{^ n ),-^ n ) -A L w(s) (.t + X(£),-£) a.e. in [0,t] 

and, in addition, that the L u ^(x + X(£ n ), —£ n ) are uniformly bounded. We thus get the 
assertion through the dominated convergence theorem. □ 
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